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Abstract — The normalized min-sum algorithm can achieve 
near-optimal performance at decoding LDPC codes. However, 
it is a critical question to understand the mathematical principle 
underlying the algorithm. Traditionally, people thought that the 
normalized min-sum algorithm is a good approximation to the 
sum-product algorithm, the best known algorithm for decoding 
LDPC codes and Tlirbo codes. This paper offers an alternative 
approach to understand the normalized min-sum algorithm. 
The algorithm is derived directly from cooperative optimization, 
a newly discovered general method for global/combinatorial 
optimization. This approach provides us another theoretical basis 
for the algorithm and offers new insights on its power and 
limitation. It also gives us a general framework for designing 
new decoding algorithms. 

I. Introduction 

LDPC codes can achieve near capacity performance for 
channel coding. One popular algorithm for decoding LDPC 
codes is so called the normalized min-sum algorithm [1], [2] 
(also referred to as the normalized BP-based algorithm). It 
is attractive for hardware/software implementations because 
it reduces the implementation complexity of the sum-product 
algorithm without losing much of its performance. The sum- 
product algorithm [3], [4], is the best known algorithm for 
decoding both Turbo codes [5] and LDPC codes [6], [7]. 

The min-sum algorithm [8] in its original form is treated 
as a generalization of the Viterbi algorithm [9] for iterative 
decoding of code realizations on general graphs. When a code 
realization is cycle-free, the algorithm is an exact solution for 
maximum likelihood decoding. When there are cycles, it is 
surprising that the min-sum decoding often works quite well 
in terms of empirical performance. However, we are lack of 
theoretical understanding about its remarkable performance in 
this case. 

The sum-product algorithm [6] can be viewed as a general- 
ization of the belief propagation (BP) algorithm developed in 
AI [10]. The min-sum algorithm is also referred to as the BP- 
based algorithm [11] where the former can be understood as 
an approximation to the later following the standard approxi- 
mation of the Max-Log -MAP [12]. Compared with the sum- 
product algorithm, the min-sum algorithm sometimes may 
have a noticeable degradation in performance. 

The normalized min-sum algorithm was proposed as a 
better approximation to the sum-product algorithm than the 
original min-sum algorithm. Simulation results show that the 



normalized min-sum algorithm can improve the performance 
of the original min-sum algorithm [1]. It can also achieve a 
near-optimal performance in many cases at decoding LDPC 
codes, very close to the sum-product algorithm. 

It might not be straightforward to understand the (nor- 
malized) min-sum algorithm if we argue that it is an ap- 
proximation to the sum-product algorithm. The sum-product 
algorithm is only an approximate algorithm for computing the 
marginal a posteriori distributions. The optimal decoding for a 
channel code, described in Shannon's information theory [13], 
is based on finding the codeword of the maximum a posteriori 
probability. The two tasks are closely related to each other, 
but not equivalent. Furthermore, we still need more theoretical 
understanding of the sum-product algorithm even though there 
are some remarkable progresses recently [16], [17], [18]. 

This paper takes different approach to understand the 
(normalized) min-sum algorithm. It will show that the al- 
gorithm can be derived as a cooperative optimization algo- 
rithm [14]. Cooperative optimization is a newly discovered 
global/combinatorial optimization method for attacking hard 
optimization problems. It breaks a hard optimization problem 
into a number of sub-problems and solves them together in 
a cooperative way. It does not struggle with local minima, 
has global optimality conditions for recognizing global op- 
tima, and offers us a complete departure from the classic 
optimization methods. With proper settings, a cooperative 
optimization algorithm has a unique equilibrium and converges 
to it with an exponential rate regardless of initial conditions 
and perturbations. 

Deriving the normalized min-sum algorithm as a coopera- 
tive optimization algorithm can offer us new insights about 
the algorithm. Following this approach, this paper attempts to 
answer a number of important questions related to the min- 
sum algorithm; 1) why can it find the optimal codeword? 2) 
whether a found codeword is the optimal one? 3) what is 
its objective function to be optimized? A general framework 
is also presented in this paper for designing new decoding 
algorithms in a systematic way. 

II. Cooperative Optimization 

Cooperative optimization is a general principle for 
finding the global optimum of a multivariate function 
E{xi,X2, . . . , Xn). It Utilizes another function in simple forms. 



such as '^{xi,X2t ■ ■ ,Xn) = ^^1^ iterativcly re- 

fines the function as the lower bound of the multivariate 
function E{x). At a given iteration k, if the lower bound 
function ^'('^^(a;) has been tightened enough so that its global 
minimum equals to the global minimum of E{x), i.e., 

min*('')(x) min£'(a;) , 

X X 

then the global minimum of E{x) is found which is the same 
as the global minimum of ^'^'"'^(x), 

argmin£'(a;) ~ argmin (x) . 

X X 

The global minimum of ^E'^'^^ (x) of the form ^E'^'^^ [xi) can 
be easily found as 

^ argmin*f^(a;0, for i = 1, 2, . . . , n . 

Xi 

Assume that the multivariate function E{x), often referred 
to as the energy function or the objective function, can be 
decomposed as the aggregation of a number of sub-objective 
functions, 

i 

Assume further that vI/C^-i) (x) of the form ^ - [xi) is a 

lower bound function of E{x), ^^''^ {x) < E{x). Let '^f'' {xi), 
for i = 1, 2, . . . , 71, be computed as follows 

■^f\xi)^ m:in{l~\k)E,{x)+\ky^w,j-^f-^\x,) , (1) 

Xi\xi ■' 

i 

where Xi the set of variables contained in Ei(x) and minXiXx; 
stands for minimizing with respect to all variables in Xi 
excluding Xi. Then the new function ^^^^{x) = Y.^ ^f^ {^^) 
is also a lower bound function of E{x). In the above equation, 
Afe and Wij (1 < «,j < n) are coefficients of the linear 
combination of Ei and ^'^'"^(xj). Afc satisfies < Afe < 1 
and uiijS satisfy Wij > and J^i ^ij — 1 for 1 < i, j < n. 

Without loss of generality, assume that all objective func- 
tions including the sub-objective functions Ei{x) are nonneg- 
ative functions. Then the cooperative optimization theory tells 
us that the lower bound function ^(x) computed by ([0 can 
be progressively tightened, 

¥°\x) < ¥^\x) <...< ¥'''>{x) < E{x) , 

when we choose the initial condition as '^'f\xi) — 0, for 
i = 1, 2, . . . , 71. 

The difference equations Q define the dynamics of co- 
operative optimization. The original minimization problem 
minx E{x) has been divided into n sub-problems of minimiza- 
tion (see Q). Those sub-problems can be solved in parallel 
in implementation. The function 'ii^^'\xi) is the solution at 
solving the ith sub-problem. The objective function of the ith 
sub-problem, denoted as Ei{x), is a linear combination of the 
original sub-objective function Ei{x) and the solutions from 
solving other sub-problems, i.e., 

eI'\x) = (1 - Afc) E,{x) + Afc ^ w,,^f\x,) . 

3 



The cooperation among solving those sub-problems is thus 
achieved by having each sub-problem compromising its solu- 
tion with the solutions of other sub-problems. Ei{x) is called 
the modified objective function for the sub-problem i. 

The coefficient Afe is a parameter for controlling the cooper- 
ation at solving the sub-problems and is called the cooperation 
strength. A high cooperation strength leads to strong cooper- 
ation at solving the sub-problems while a lower cooperation 
strength leads to weak cooperation. The coefficients Wij con- 
trol the propagation of the sub-problem solutions '^i{xi) in 
the modified objective functions Ei{x) (details in [14]). They 
are so called the message propagation parameters. 

The function '^\'^\xi) can be understood as the soft de- 
cision of assigning the variable Xi at minimizing Ei{x). 
The most preferable value for variable Xi at iteration k is 
argminj;^ (a;,). 

Given any variable, say Xi, it may be contained in several 
sub-problems. At each iteration, Xi has a value in the optimal 
solution for each of the sub-problems. Those values may 
not be the same. If all of them are of the same value, 
denoted as Xi, we say that all the sub-problems reach a 
consensus assignment for variable Xi. If all sub-problems reach 
a consensus assignment for each variable at some iteration, 
we say that a consensus solution is reached at the iteration. 
Consensus solution is an important concept of cooperative op- 
timization for defining global optimality conditions. Normally, 
a consensus solution is the global optimum, guaranteed by 
theory. The following theory [14] offers one global optimality 
condition based on the concept of consensus solution. 

Theorem 2.1: Assume that the difference equations Q for 
cooperative optimization reaches its equilibrium, denoted as 
{^"^{xi)) (1 < i < 77,), i.e., {^°°{xi)) is a solution to the 
difference equations Q. If a consensus solution x is found in 
this case, then it must be the global optimum of E{x), x = x* . 

It has been shown in [14] that the cooperative optimization 
algorithm defined by the difference equations Q has many 
important computational properties not possessed by conven- 
tional optimization algorithms. Given a constant cooperation 
strength A^ = A and the propagation coefficients Wij, the algo- 
rithm has one and only one equilibrium. It always converges to 
the unique equilibrium with an exponential rate regardless of 
initial conditions and perturbations. Mathematical analysis also 
shows that when A ^ 1, the lower bound function computed 
by the difference equations ([0 tends to have a global minimum 
approaching the global minimum of the original function as 
the iteration proceeds. Whenever those two global minimums 
touch each other, the global minimum of the original function 
is found. 

Hence, the cooperative optimization is stable (unique equi- 
librium), fast (exponential convergence rate), and robust (in- 
sensitive to initial conditions and perturbations). Unlike con- 
ventional optimization methods, it does not struggle with local 
minimums and it knows when to stop search because of the 
global optimality condition. Details about these together with 
the theoretical investigation of cooperative optimization are 
provided in [14]. 



Let ■<^f\x{)/{l - \k) => "^T'i^^)^ the difference equa^ 



tions Q can be rewritten in a different form, 

■^f\x,) = min E, + \kY\ 



{^,) , (2) 



With certain way of decomposing E{x), certain settings 
of the cooperation strength Afe and the message propagation 
parameters Wij, the normaHzed min-sum algorithm can be 
derived from the difference equations Q of cooperative opti- 
mization. 

III. Constructing More Powerful Cooperative 
Optimization Algorithms 

Describing cooperative optimization in the language of 
mathematics enables us to make a generalization of the ba- 
sic cooperative optimization algorithm (see Difference Equa- 
tion O). One way to generalize it is to take more complicated 
forms of the lower bound functions other than the simple one 
Oils of them is to break ^i(xi) into several 
pieces as follows 

^ i i-^i ) ^ ^ ii ('^'i ) ' ' ■ ■ ■ ; ^iiv^ ('^'i ) ' 

Consequently, the lower bound function takes the following 
form 

E^{xi,X2, ■ ■ ■ ,Xn) =^^^ij{Xi) . (3) 

i 

Such a lower bound function is called the fragmented unary 
lower bound function. 

Let {Eij{x)} be a decomposition of E{x), i.e.. 



» 3 

Assume that there is one sub-objective function Eij{x) for 
each 'i>ij{xi). Correspondingly, the difference equations of the 
cooperative optimization become 

i' 3 

where Xij is a set of variables containing those in Eij{x). 

Our simulation has demonstrated that cooperative optimiza- 
tion algorithms based on the fragmented unary lower bound 
function are often more powerful at decoding LDPC codes 
than those based on the simple form. 

IV. LDPC Decoding as Combinatorial Optimization 

LDPC codes belong to a special class of linear block codes 
whose parity check matrix H has a low density of ones. For 
a LDPC code over GF{q), its parity check matrix H has 
elements hmn defined over GF{q), hmn G GF{q). Let the 
code word length be TV (the number of symbols), then H 
is a M X N matrix, where AI is the number of rows. Each 
row of H introduces one parity check constraint on input data 
X = {xi,X2,-.-,xn), i.e.. 



N 
n=l 



0, for m — 1,2, 



,M 



Putting the m constraints together, we have Hx^ 
Let function /„(a;„) be defined as 

fn{Xn) = -\np{Xn/yn) , 



0. 



(5) 



where p(x„/t/„) is the conditional distribution of input data 
symbol n at value Xn given the output data symbol n at value 
Vn- /n(0) -/„(a;„), which is equal to ln(p(j:„/y„)/p(0/y„)), 
is the log-likelihood ratio (LLR) of input data symbol n at 
value Xn versus value 0. 

In those notations, the maximum likelihood decoding can 
be formulated as a constrained combinatorial optimization 
problem. 



mm 



E 

n=l 



fniXn) S.t. iJx^ = 



(6) 



The function to be minimized in (|6j is called the objective 
function for decoding a LDPC code. The decoding problem 
is, thus, transferred as finding the global minimum of a multi- 
variate objective function. 

Let X be the set of all variables. Given the mth constraint 
be HjyiX^ — 0, let X^^ be the set of variables corresponding 
to the non-zero elements in i?™, i.e., 

Xra — {•^n\^mn 7^ 0} • 

Let fxmiXra) be a function defined over X,,, as 

0, if Hr.^x'^ = 0; 
oo, otherwise. 



fx^ {Xm) 



(7) 



fx^{Xm) is called the constraint function representing the 
mth constraint. Using the constraint functions, the decoding 
problem (|6j can be reformulated as an unconstrained optimiza- 
tion problem of the following objective function. 



M N 
m— 1 71—1 



(8) 



Such a combinatorial optimization problem is, in general, NP- 
hard. 

V. Deriving the Normalized Min-Sum Algorithm 

In the following discussions, we differentiate unary con- 
straints from higher order constraints in notations by using 
symbol /„(a;„) for unary constraint on variable Xn and 
/x,„ (Xm) for a constraint of an order higher than one. When 
constraints are mentioned, they are referred to non-unary 
constraints and unary constraints will be explicitly declared. 

Conventionally, a LDPC code is represented as a Tanner 
graph, a graphical model useful at understanding code struc- 
tures and decoding algorithms. A Tanner graph is a bipartite 
graph with variable nodes on one side and constraint nodes 
on the other side. Edges in the graph connect constraint 
nodes to variable nodes. A constraint node connects to those 
variable nodes that are contained in the constraint. A variable 
node connects to those constraint nodes that use the variable 
in the constraints. Constraint nodes are also referred to as 
check nodes. During each iteration of the min-sum algorithm. 



messages are flowed from variables nodes to the check nodes 
first, then back to variable nodes from check nodes. 

To follow the notation used in the literatures of coding, we 
change the symbols for indices from i and j to m and n. 
Let M{ra) be the set of variable nodes that are connected to 
the check node m. Let M{n) be the set of check nodes that 
are connected to the variable node n. Let symbol '\' denotes 
the set minus. N{m) \ n denotes the set of variable nodes 
excluding node n that are connected to the check node m. 
M{n) \ m stands for the set of check nodes excluding the 
check node m which are connected to the variable node n. 

To derive the normalized min-sum algorithm as a coopera- 
tive optimization algorithm, we choose the fragmented unary 
function (see (|3}) as the form of the lower bound function. 
For each variable n and then for each constraint containing 
the variable, m G AA{n), we define a component function 
'^nmixn)- The Summation of all those component functions 
is the form of the lower bound function. 



,Xn) 



N 

E 

n=l meM{n 



Let the decomposition of the objective function E{x) for 
a LDPC decoding problem be {i?„,,„(a;)}. Functions Enm{x) 
are sub-objective functions. There is one sub-objective func- 
tion Enm{x) for each 5'„m(x„). The overall objective function 
is 

Enm{x) . 

Different decompositions lead to different cooperative opti- 
mization algorithms. The decomposition leading to the nor- 
malized min-sum algorithm has the following form 

Enm{x) = fxAXm) + Y fn'M- (9) 
n GA/'(m)\n 

It contains the mth constraint fx„^{Xm), all the unary con- 
straints on the variables in Xm except the one on a;„. 

Substituting (|9j into the difference equations @ for 
the fragmented unary lower bound function and choosing 
^fcW'nmn'm' = "fc Constant), we have 

n £j\f{m)\n 

(10) 

where 



ra GA4(n )\m 



^^^-}^\x^) . (11) 



For binary LDPC codes, all a;„s are binary variables, the 
unary constraint fn{xn) defined in (|5} can be rewritten as 

/„(!) = In and /„(0) = . (12) 

p(xn = l/y„) 

This change does not have any impact on the objective func- 
tion ^ of decoding except offsetting the objective function by 
a constant. We can also offset ^nlixn) in (d by *iU(0). 



This change also does not have any impact on the optimization 
results. Putting these two changes together, the difference 
equations JlOt can be simplified to 

vI'W(0) = 0, and 



n' e7V(m)\n 



sgn(Z^r min \Z^,'>(1)\. 

«'eAA(m)\n " 

(13) 

Eq. (I13> is the check node update rule of the normalized min- 
sum algorithm and Eq. il l\ is the variable node update rule. 
The normalized min-sum algorithm is, thus, derived from the 
difference equations of cooperative optimization. 

Difference Equation dlOt defines an optimization algorithm 
more general than the normalized min-sum algorithm. It is 
suitable for any kind of variables and any kind of constraints. 
The normalized min-sum algorithm defined by ( I13> and il l\ is 
a special instance of the algorithm, applicable only to boolean 
variables and parity check constraints. By understanding the 
underlying principle of the normalized min-sum algorithm, we 
make it possible to derive more general decoding algorithms. 

VI. Objective Function of the Min-Sum Algorithm 

When we derived the normalized min-sum algorithm in the 
previous section, we did not check the objective function it 
tries to minimize. From the cooperative optimization theory, 
the objective function can be obtained by summing up all sub- 
objective functions. In the case of the normalized min-sum 
algorithm, the objective function is 

E E (/^™(^™)+ E fn'iXn') 
n meM{n) \ n'eAf{m)\n 

If /x„(^m)s are parity check constraints (0, we can drop 
fXmiXm) in the above objective function and rewrite it as 

EE E fn'i^n), S.t.Hx^ = 0. 

n meM(n)n' eN'(m)\n 

This objective function is, in general, not proportional to 
Tim fn{xn) cxccpt of a fcw cascs, e.g., regular LDPC codes. 
A regular LDPC code has the property that all variable nodes 
have the same degree and all the check nodes also have the 
same degree. Otherwise, it is called an irregular LDPC code. 

Let dy{n) be the degree of the variable node n. Let dc{m) 
be the degree of the check node m. Using these notations, 
the objective function of the normalized min-sum algorithm 
becomes 



Anfnixn)-, where 



An= Y (4M-1), forn= 1,2,...,7V . (14) 

meM(n) 

The above objective function is proportional to the desired 
objective function J^n fn{xn) if and only if all A„s have the 
same value for all ns. 



Assume that all constraints in a LDPC code have the same 
degree, dc, then An in Formula il4\ becomes 

(4 - forn= 1,2,...,7V . 

Hence the weight for each variable in the objective function 
is proportional to the degree of the variable. The objective 
function of the normalized min-sum algorithm is biased in 
this case. It weights bit variables of higher degrees more than 
the variables of lower degrees. That explains why we have 
experienced less decoding errors for bit variables of higher 
degrees in applying the normalized min-sum algorithm at 
decoding LDPC codes in practice. 

For a regular LDPC code, dc{m) — dc (a constant) and 
dy{n) = dy (a constant). In this case, A„ — {dc — l)dv for 
all n, and the objective function of the min-sum algorithm 
is proportional to J2n fni^n), which is the desired objective 
function for the maximum likelihood decoding of the code. 

VII. General Framework for Constructing 
MiN-SuM Algorithms 

Viewing the normalized min-sum algorithm as an instance 
of cooperative optimization lays out several new ways to 
design new decoding algorithms. One way is to select different 
forms of the lower bound functions besides the simple form 
and the fragmented form offered in this paper. Another way is 
to explore different decompositions of the objective function 
associated with a decoding problem. The third way is to use 
different settings of the cooperation strength and message 
propagation parameters Wij. The decomposition presented in 
this paper is simple and direct. It is only one of many possible 
ways of decomposing an objective function. Correspondingly, 
representing an objective function using a Tanner graph, 
different decompositions of the objective function correspond 
to different ways of decomposing the Tanner graph into sub- 
graphs. Each LPDC code represented as a Tanner graph 
may also have its own unique graphical structure. Special 
decomposition can also be explored to maximize the power 
of cooperative optimization at decoding the code. 

VIII. Conclusions 

This paper has shown that the normalized min-sum algo- 
rithm is the simplification of a cooperative optimization algo- 
rithm for the special case of boolean variables and parity check 
constraints. A generalized min-sum algorithm for decoding 
LDPC codes over GF{q) , for any q > 2, and any form of 
check constraints are also offered in this paper (see Eq. I10> . 

The normalized min-sum algorithm can be understood as 
finding the global minimum via a lower bounding technique. 
To decode a LDPC code, it deploys a function of a simple 
form which is a lower bound function to the objective function 
associated with the decoding problem. To find the global 
optimum of the original objective function, it progressively 
tightens the lower bound function until the global minimum 
of the lower bound function reaches the global minimum of 
the original objective function. 



A consensus solution found by a cooperative optimization 
algorithm is in general the global optimum. It offers a general 
criterion for the normalized min-sum algorithm to identify the 
optimal codeword and to terminate its search process. The 
criterion is that each variable of the problem has the same best 
assignment in minimizing the modified sub-objective functions 
defined by the cooperative optimization (see Eq. I10> . 

It has also been shown that the normalized min-sum al- 
gorithm has a biased objective function with higher weights 
for those variables of higher degrees. The objective function 
becomes the desired, unbiased objective function when a 
LDPC code is regular. 
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